4 research outputs found
Neural Distributed Compressor Discovers Binning
We consider lossy compression of an information source when the decoder has
lossless access to a correlated one. This setup, also known as the Wyner-Ziv
problem, is a special case of distributed source coding. To this day, practical
approaches for the Wyner-Ziv problem have neither been fully developed nor
heavily investigated. We propose a data-driven method based on machine learning
that leverages the universal function approximation capability of artificial
neural networks. We find that our neural network-based compression scheme,
based on variational vector quantization, recovers some principles of the
optimum theoretical solution of the Wyner-Ziv setup, such as binning in the
source space as well as optimal combination of the quantization index and side
information, for exemplary sources. These behaviors emerge although no
structure exploiting knowledge of the source distributions was imposed. Binning
is a widely used tool in information theoretic proofs and methods, and to our
knowledge, this is the first time it has been explicitly observed to emerge
from data-driven learning.Comment: draft of a journal version of our previous ISIT 2023 paper (available
at: arXiv:2305.04380). arXiv admin note: substantial text overlap with
arXiv:2305.0438
Distributed Deep Joint Source-Channel Coding with Decoder-Only Side Information
We consider low-latency image transmission over a noisy wireless channel when
correlated side information is present only at the receiver side (the Wyner-Ziv
scenario). In particular, we are interested in developing practical schemes
using a data-driven joint source-channel coding (JSCC) approach, which has been
previously shown to outperform conventional separation-based approaches in the
practical finite blocklength regimes, and to provide graceful degradation with
channel quality. We propose a novel neural network architecture that
incorporates the decoder-only side information at multiple stages at the
receiver side. Our results demonstrate that the proposed method succeeds in
integrating the side information, yielding improved performance at all channel
noise levels in terms of the various distortion criteria considered here,
especially at low channel signal-to-noise ratios (SNRs) and small bandwidth
ratios (BRs). We also provide the source code of the proposed method to enable
further research and reproducibility of the results.Comment: 7 pages, 4 figure